The Complexity of Data-Driven Norm Synthesis and Revision

Norms have been widely proposed as a way of coordinating and controlling the activities of agents in a multi-agent system (MAS). A norm specifies the behaviour an agent should follow in order to achieve the objective of the MAS. However, designing norms to achieve a particular system objective can be difficult, particularly when there is no direct link between the language in which the system objective is stated and the language in which the norms can be expressed. In this paper, we consider the problem of synthesising a norm from traces of agent behaviour, where each trace is labelled with whether the behaviour satisfies the system objective. We show that the norm synthesis problem is NP-complete

Evaluating Classifiers in SE Research: The ECSER Pipeline and Two Replication Studies

Context: Automated classifiers, often based on machine learning (ML), are increasingly used in software engineering (SE) for labelling previously unseen SE data. Researchers have proposed automated classifiers that predict if a code chunk is a clone, if a requirement is functional or non-functional, if the outcome of a test case is non-deterministic, etc. Objective: The lack of guidelines for applying and reporting classification techniques for SE research leads to studies in which important research steps may be skipped, key findings might not be identified and shared, and the readers may find reported results (e.g., precision or recall above 90%) that are not a credible representation of the performance in operational contexts. The goal of this paper is to advance ML4SE research by proposing rigorous ways of conducting and reporting research. Results: We introduce the ECSER (Evaluating Classifiers in Software Engineering Research) pipeline, which includes a series of steps for conducting and evaluating automated classification research in SE. Then, we conduct two replication studies where we apply ECSER to recent research in requirements engineering and in software testing. Conclusions: In addition to demonstrating the applicability of the pipeline, the replication studies demonstrate ECSER’s usefulness: not only do we confirm and strengthen some findings identified by the original authors, but we also discover additional ones. Some of these findings contradict the original ones

A Framework for Modeling Human Behavior in Large-scale Agent-based Epidemic Simulations

Acknowledgements We thank Cuebiq; mobility data is provided by Cuebiq, a location intelligence and measurement platform. Through its Data for Good program, Cuebiq provides access to aggregated mobility data for academic research and humanitarian initiatives. This first-party data is collected from anonymized users who have opted-in to provide access to their location data anonymously, through a GDPR and CCPA compliant framework. To further preserve privacy, portions of the data are aggregated to the census-block group level. For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.Peer reviewedPublisher PD

Requirements Classification with Interpretable Machine Learning and Dependency Parsing

Requirements classification is a traditional application of machine learning (ML) to RE that helps handle large requirements datasets. A prime example of an RE classification problem is the distinction between functional and non-functional (quality) requirements. State-of-the-art classifiers build their effectiveness on a large set of word features like text n-grams or POS n-grams, which do not fully capture the essence of a requirement. As a result, it is arduous for human analysts to interpret the classification results by exploring the classifier's inner workings. We propose the use of more general linguistic features, such as dependency types, for the construction of interpretable ML classifiers for RE. Through a feature engineering effort, in which we are assisted by modern introspection tools that reveal the hidden inner workings of ML classifiers, we derive a set of 17 linguistic features. While classifiers that use our proposed features fit the training set slightly worse than those that use high-dimensional feature sets, our approach performs generally better on validation datasets and it is more interpretable

Data-Driven Supervision of Autonomous Systems

Modern software systems execute in increasingly dynamic settings, and their objectives are in constant motion. In order to preserve their adequacy and effectiveness within an evolving environment, software and its requirements need to adapt to change. In this dissertation, we propose a data-driven supervision framework for the automatic run-time revision of requirements, so to ensure the achievement of system-level objectives in dynamic settings. We focus on the supervision of multi-agent systems (MASs), collections of interacting autonomous agents, such as autonomous cars on smart roads. In multi-agent systems, agents’ internals are typically unknown to the other agents and to the MAS designer. Norms are often employed as a means for controlling and coordinating the agents' behavior without over-constraining their autonomy. We use norms to characterize requirements for the behavior of the agents in the system, and we use sanctions as a deterrence mechanism to discourage agents from violations. The proposed supervision framework employs a general architecture for system self-adaptation, described as a closed control-loop. At run-time, the system is monitored and execution data is collected in different operating contexts. The collected data is used to learn statistical correlations between the achievement of the system's objectives and the satisfaction of the requirements in the different operating contexts. The learnt information is applied to automatically assess the validity of the assumptions made at design-time, and to automatically synthesise new requirements and sanctions when there is evidence that the current ones are not effective

Data-Driven Supervision of Autonomous Systems

Modern software systems execute in increasingly dynamic settings, and their objectives are in constant motion. In order to preserve their adequacy and effectiveness within an evolving environment, software and its requirements need to adapt to change. In this dissertation, we propose a data-driven supervision framework for the automatic run-time revision of requirements, so to ensure the achievement of system-level objectives in dynamic settings. We focus on the supervision of multi-agent systems (MASs), collections of interacting autonomous agents, such as autonomous cars on smart roads. In multi-agent systems, agents’ internals are typically unknown to the other agents and to the MAS designer. Norms are often employed as a means for controlling and coordinating the agents' behavior without over-constraining their autonomy. We use norms to characterize requirements for the behavior of the agents in the system, and we use sanctions as a deterrence mechanism to discourage agents from violations. The proposed supervision framework employs a general architecture for system self-adaptation, described as a closed control-loop. At run-time, the system is monitored and execution data is collected in different operating contexts. The collected data is used to learn statistical correlations between the achievement of the system's objectives and the satisfaction of the requirements in the different operating contexts. The learnt information is applied to automatically assess the validity of the assumptions made at design-time, and to automatically synthesise new requirements and sanctions when there is evidence that the current ones are not effective

Requirements-driven supervision of socio-technical systems

Modern software systems are characterized by ever-changing goals and requirements. Such systems operate in an environment that is dynamic, open, partly known, unpredictable. New goals arise and others are dropped, due to changes in stakeholders' needs and priorities, government regulations, technology. Despite this dynamism, systems should meet their goals and comply with the evolving requirements. While several self-adaptation mechanisms have been proposed in the literature, they cannot be fully applied for socio-technical systems that involve autonomous (thus, non-controllable) components. This project aims at designing and developing a runtime requirements supervision framework that monitors the execution of socio-technical systems, evaluates their behavior against the overall goals and intervenes by deciding how to revise requirements when adaptation is not possible